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ABSTRACT 



An image processing system redistributes imag e lumi- 
nan ce values in an image represented in u'v'L* space t o^ 
achiev e a near-optimal fit of luminance data tQ.a sfflilatlle 
dynamic ran^e and t o enhance the shape and detail^ f 
ihe resultant im age. Redistribution may be accom- 
plislied by passing the image through a filter having a 
peak response at a spatial.frequency at which the human 
eye is highly sensitive, and then equalizing a cumulative 
histogram of filtered pixels. An image may be trans- 
formed to compensate for tone compression which 
accompanies printing processes. 

26 Claims, 13 Drawing Sheets 
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piintmg characteristics and by manipulating the image 

AUTOMATED TONE CORRECTION APPARATUS characteristics with a set of easily understood icons. 

AND METHOD USING FILTERED HISTOGRAM Using the Prophecy system, an operator can perform 

EQUALIZATION many image manipulation operations such as tone and 
PPT ATPn APPT Tn ATTr^Mc ^ contrast correction, cast removal, image sharpening and 

Kti-A 1 tiu Ai'f i,iCA 1 lONb blumng, silhouetting, selective color changes, cloning, 

Related U.S. patent application Scr. No. 07/983.154, masking, image rotation, and OPI substitution, 

entitied *1mage Processing Apparatus and Method with The electronic image processed by the workstation 

Intelligent Pixel Measurement Cursor" and Scr. No. consists of a two-dimensional array of picture elements 

07/717,099, entitled "A Color Image Processing Sys- (pixels). Each pixel may be represented in any of a vari- 

tem for Preparing a Composite Image Transformation ety of color notations or "color spaces'*. For example, 

Module for Performing a Plurality of Selected Image the RGB color space represents pixel according to the 

Transformations,*' are incorporated by reference. relative contributions of three primary colors, red, 

BACKGROUND OF THE INVENTION u ^f^' ^ ^^"^^l ™* notation is commonly used by 

" color monitors smce the three parameters (R, G, and B) 

The present invention relates to image processing correspond to the mechanism by which the monitor 
systems, and in particular to image processing systems generates color. More specificaUy, each pixel of the 
for tonal correction of images. monitor's display contains a red phosphor, a green 

Image processing systems typically mclude an input phosphor, and a blue phosphor. To generate a color 
device for generatmg an electronic representation of an 20 ^^^^ ^ ^ ^^^^^ ^ ^^^^^ stimulates 

miage. The mput device provides die e ectronic image each primary phosphor with an intensity determined by 
repres^tation to a computer worksution which pro- the approprkte pi^'s corresponding RGB v^ 
cf^ theimagemaccordan« withaus^^smstruc^^^ SiiSlariy, the CMYK space reprLnts color using 
and forwards the processed image to a high resolution ft,„, ^o^aWoc tlt v *~^^ ^*'**=^""* wiui ua^g 
monitor for display. Tlie user interacts wx^ the work- 25 'XtivHS^ctiS/i^ 
sution, repeatedly instructing it to adjust the electronic ^^^^"^ (subtract^ye) contobution^ of the colorants, 
image unTthc monitor displays a dLired image. TTie magenta, yeUow and black. This representation is 

user can also generate a hard copy of the ii^ge by ^^^^yj^ by printing defaces smce each parame. 
instructing tiic workstation to provide the processed Y and K determmes the amount of a colorant 

electronic image to a selected printing device. 30 ^f'^r ^ ^^^^ ^ the printer in producing a 

An exemplary image processing system is shown in Notations such as RGB and CMYK are 

FIG. t In this Figure, an image is input to the system by scannmg devices and image printing 

scanning hardcopy with scanner 10. Scanner 10 may be <J«vices, respectively, since each parameter of the space . 
any suitable color or black-and-white scanner, such as a closely corresponds to a physical mechanism by which 
Macintosh-compatible scanner capable of producing 35 devices measure and generate color. 
24-bit RGB files. Alternatively, the image may be input . These spaces may net be well-suited for processing 
through a disk storage unit 12 such as the Kodak Pho- images. For example, as shown in FIG. 2, the three 
toCD system. Images may also be input directiy from parameters R, G and B define a three dimensional space 
video cameras and other image sensing devices. point within the space, corresponding to a 

The scanner may generate an image file using a suit- 40 ^<iue color combination. At various points within the 
able workstation 14 such as a Macintosh U/fx or Macin- space, a selected change in the values of the parameters 
tosh Quadra. The computer typically has a math co- result in a commensurate change in the per- 

processor chip. Using the workstation, the overall ceived color. For example, at a location in the space 
image may be positioned, scaled and cropped with suit- having a high R parameter value, increasing the param- 
able page compositian software such as Aldus Page- 4S R by n units might yield little perceived change in 
Maker, Quark XPress, or Letraset Design Studio. color. Yet, at another point in the space having a rela- 

The image also may be processed on an appropriate tivcly low R parameter value, an increase of the same n 
imaging station 16 such as a Kodak PCS 100 worksta- ^iiuts yields a dran:iatic change in the perceived color, 
tion. which is built around a Macintosh Quadra 900 Accordingly, it may be difficult for a user to manipulate 
platform equipped with a Kodak Precision Color 50 the primaries R, G, B to achieve a desired change in 
Transfonn Engine. The PCS 100 also uses a 24.bit color color. 

graphics card such as the SuperMac Thunder/24 card In response to this problem, a variety of perceptually- 
and color display calibration hardware. Using the based spaces have bosn proposed for definmg colors in 
PCS 100, tlie operator can make use of the advanced terms of parameters which more closely correspond to 
photo editing toolset of the Adobe PhotoShop system. 55 the manner in which humans perceive color. The most 
The image processed by imaging station 16 may be prominent perceptually-based standards for such repre- 
sent to a proofer 20 for hardcopy output. The proofer sentation are collectively referred to as the CIE system, 
may be any suitable compatible with the imaging sta- which is promulgated by the International Commission 
tion. Preferably, data interchange between the imaging on Illumination. 

station and the proofer is accomphshod in a four color 60 The 'Si'v'L*" space, for example, is a three dunen- 
separation format such as DCS CMYK format, EPS sional space. The chromaticity of each point in this 
CMYK format, or Sdtex CT format. space is uniformly characterized by the parameters u' 

While an image may be processed as above, it may and v'. The third parameter, L*, denotes perceptually 
also be sent to a suitable publishing system 18 such as uniform variations in the lightness, (e.g., L*=0 is black, 
the Kodak Prophecy system, which is built around an 65 L*=fiill-scale is white). Luminance values in a uVL* 
image accelerator-equipped Sun SPARC platform. The representation will be noted by the symbol "IT, 
publishing system allows the operator to. "soft-proof ' To process an image in the u'vX space, the worksta- 
the image by correcting the image to compensate for tion maps each point u'o, v'o, Lq in the space to a new 
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point u'l, v'l, Li. For example, if the user wishes to ferred exemplary embodiments, taken in conjunction 

display die image on a monitor, he may need to adjust with the accompanying drawings, of which: 

the image to compensate for bright lighting conditions FIG. 1 is a diagram of an exemplary image processing 

of the room. Accordingly, the user may select a trans- system. 

form which maps each point u'o, v'a Lo to a new point 5 FIG. 2 is a three-dimensional graph of a gray scale in 

having the same , values u'q, v'o but having a greater a RGB space. 

luminance value Li. FIG. 3 is a block diagram of an nnage processing 

One technique for tonal correction applied in the system embodying the present invention, 

prior art consists of adjusting the pixels in an image in flG, 4 is a histogram of luminance values in a sample 

such a way that the shadows and highlights within the 10 maage. 

image are mapped to specific target values. These meth- FIG- 5 is a cumulative histogram of the histogram of 

ods are useful in adjusting the overall brightness and FIG. 4. 

dynamic range of an image; however, they are not ef- F^G- ^ is a remapped histogram of luminance values 

fective at adjusting midtone gray levels. ^ "^S® ^^G- 

U.S. Pat No. 4,729,016 to Alkofer describes tech- FIG. 7 is a cumulative histogram of the lummance 

niques for tonal correction usmg histograms. The Al- values of the remapped histogram of FIG. 6. 

kofer technique is based on two assumptions. First, an ^J^^^ ^ph of a seven-pomt filter used for the 

optimal tone reproduction curve for a natural image histo^am equahzadon fimction. 

should remove aU tonal remappings performed by the ffl^^*^>?Jl^o^^^ frequency response of the 

capture process, thereby restoring the tonality of the fr^i 

original scene. Second, the distribution of lummance 10 shows a neutral gamut compression curve 

values on edges within a natural scene should be Gauss- ac^rdmg to the present mvenuo^ 

ian. Given these assumptions one could measure the V tPI ^^tr^^B remappmg of lumi- 

actual tonal distribution on edges in an image, compare ^ ^ lummance values for a 

that distribution to a Gaussian distribution, and use the ^^i^e * . i i r 

result ofthe comparison to "back out" the effe«:ts of the . -S' ^ operator contrd panel for 

tu mcess. iiutializmg a tone correction accordmg to the present 

capnireprocos. » t- * invention. 

To this end. the Alkofer technique gooerates histo- nO. 13 is a graph iUustrating a correction curve, and 

grams from a sample of tone and conteast value^ The 3^ ^ automatic to^al modified curve specified by the 

histogram producmg the most normal tone distribution control panel in FIG 12 r j 
is chosen for the transformadon function, and the func- 

tion is applied to the grayscale value Cm a black-and- DETAILED DESCRIPTION OF THE 

white system) or to each component in the color space PRESENTLY PREFERRED EXEMPLARY 

Cm a color system). 35 EMBODIMENTS 

In many cases one or both of these assumptions are should be understood at the outset that the present 

not vabd. Not all edg^ m aU scenes have mherently invention is widely appUcable in any of a variety of 

Gaussian lummance distributions. Furthermore, m electronic systems which manage images. The inven- 

many color editing appUcations the goal is the produc- tion will be described using lummance in a uVL color 

tion of a pleasing result rather than the recovery of the 40 space as a measure of tone, however, the invention may 

original scene appearance. ais^ measures of tone, such as brightness or 

SUMMARY OF THE INVENTION other basis of a color space. 

The invention may also be embodied as part of a 

It IS an object of this mvention to provide an image system as illustrated m FIG. 3. A workstation 30, such 

processing system having tonal transformation and cor- 45 as a Macintosh Il/fe or Quadra con^uter, is equipped 

lection capabiHties that differ from prior art systems by ^th a 24-bit gr^hics card 32 and a color monitor 34. 

not relying on the same assumptions about how tone Standard Macintosh Toolbox routines are used to dis- 

should be distributed in an image. piay pictures on the color screen. Standard Macintosh 

It is a further object to correct tone with nunimal menus allow the operator to control the operation of the 

assumptions about the nature of the image being pro- 50 machine, and a tonal transformation as described below 

cessed. may be embodied as an application program. 

It is yet another object to utilize a wide range of The system may also include capability for imple- 

information present in an image to be processed and to menting Precision Transforms of a variety described in 

produce pleasing results in a wide variety of images. copending U.S. patent application Ser. No. 07/717,099, 

The above objects are achieved by providing an 55 entitled "A Color Image Processing System for Prepar- 

image processing system which redistributes tone m an ing a Composite Image Transformation Module for 

image to enhance features having a spatial frequency Performing a Plurality of Selected Image Transfonna- 

response in a range in which , the human eye is most tions" (incorporated here by reference). Color manage- 

sensitive. Redistribution may be accomplished by linear ment tools and a Color Transform Engine of the types 

remapping, histogram equalization, semi-automatic his- so described in Kodak Precision CMS VAR User's Guide 

togram equalization of normalized filtered luminance (incorporated by reference and available from Kodak 

data, and automatic histogram equalization of normal- Electronic Printing System, Inc., Billerica, Mass.) are 

ized filtered luminance data. preferred when system image processing speed is an 

BRIEF DESCRIPTION OF THE DRAWINGS ^^T^lI^^'^^ • u a- ^ • .t. 

65 The operator begins by reading a picture mto the 

These and other objects and advantages of this inven- system from a scanner 36 or other input device which 

tion will become apparent and more readily appreciated produces an electronic image representation. The image 

from the following description of the presently pre- is displayed on the color monitor 34. In order to make 
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areuiate color judgmmte about the picture, the dis- (1) the image luminance component is passed through 
Eiayejmagek mo^ed by a m^ftfflnc^ g rationRia^ a spatial filter having a peak response in the region 
sion Transform wiucn compensates lor the characlcHS— where the human eye is most sensitive; 
fa cs ot me monitor 34 . (2) a normal histogram ofthe luminance component is 

lie operator also has the option of selecting an out- S computed; 
put gamut transformation and associating it with the (3) a wrighted histogram is computed in which points 
image. The ou^t gamut transform corresponds to a with a higher filtered value make a greater contribution: 
prmting device 35 and modifies the displayed image to (4) the weighted histogram is cumulated; and 
reflect prmter characteristics. In this way, what the (5) the cumulated histogram is linfariyed 
operator sees on the display closely represenU what the 10 The purpose of the filtering step is to produce a tonal 
operator will get when prin&ig the image. remapping which tends to increase the contrast in areas 

The operator can select from a pull down menu a of the picture containing a large amount of medium 
tonal teansfoim of thepresent invention as described in sized detail or medium-sharpness edges. The eye is most 
more detafl below The system appLes the transform sensitive to medium sized detail and therefore tends to 
and returns a modified image wWch may then be dis- 15 be pleased by pictures in which such detail is easy to 
played, stored, transmitted or subjected to fiirther prcv see. In terms of frequency response, this requirement 
J J , . . . means that the filter response should peak at medium 

1,,^ Wo ? w*^ °^ ^ frequencies and attenuatTat high and iWfrequencies. 

helpful to understand hstogram eqiMJization Ksto- To be more quantitative, the eye's respon^peaks at 
gramequahzationmplementsar^listnbutionofbnght- 20 approximately 2.5 lines/mm if an objectTbeing^^^ 
ness values such that a cumulative histogram of the ^t 15 inches. The scamiers commonly used by 
Srrfe^to'SoT:^" ^^^^-^ tend to Lnple U^Tt 

eieT^'^vS^^^^.^^^^'^.j^^r.'^ S?o^t5S';Srrrmr£iiir^fs 

cessed can be v«nah^ as shown m FIG. 4. For each 25 frequency LpoL at Ibout 1/5 the sSS fe^ 
lummance evel L„„ tf« number of p«els at that lutpi- ^ 'of clwse, one can readily Lltott^fflto 
nance level is counted. These counts are plotted with ^ 4 othTscamung wtT^ ^ 
the Li« values on the abscissa (honzontal) axis and the CTr^co— j«-t. -.1.1 ^ 

counts on the ordinate (vertical) a«s. thaeby pr<^uc- " '"'^v .«"^We jev«-pomt filter and 

ing a graph as shown in FIG. 4 ^ ^ 30 T ^^^^^ characteristic. A wider filter would allow 

A c^dative iZinance Ljtogram of the sample [""^t^^ontrolov^ the shape ofthe transform. b« 
image may be visualized as shown iuFIG. 5. For e^ ^. T^J"^^"^ computational tmie. A fifteen 
lumLance level Lm., the number of pixels at or bdtow ^"^^^^^ ? •^^t^'^' « 

that luminance levd is counted. Th«! ci^S^ " °° ^! ^ • 

counts are plotted with the Ui, values on the abscissa 35 • ^^^'V?*^ ^ implemented in an 

«ds and the cumulative counS on the ordinatT^ ZSLF'TT"^ workstation m a pi^ely hardware 
thereby producing a cmnulative graph as in nO. s!^ mplementation or by usmg a software filter, thus pro- 
Histogram equalization tnmsfonm pixel luminance * frequency response result as shown in 
values so that the resulting cumulative histogram is ~ '., . 

hncar. For example, if half of the pixels in the image 40 a-ttempis to give the greatest 

have luminance values at or below luminance l.I=H ^ bnghtaess patterns that are at the peak 

as shown in FIG. 5 and H is not the middle ofthe lumi- ^^""^y response of the human eye. Conceptually, 
nance range OuminanceLto=M), then an pixels having o=e «n "nagme an Effect ' fimction as: 
luminance H are converted to have a luminance M. Eaec,o..y)=Fater Hcnuc m 

Other pixels are nuqpped accordingly. 45 ^ ' 

iJ^J^^^f!^^'^'f^^'^'^^^'^ ^ convolution operator, Filter is the 

Wormaton to the image Imnmance values L « above-described filter function ^ Pic^ire is an array 

^own FIG. 6, and the cumuhUxve histogram is representing the image pixels in uVL space Becau^ 

diown m FIG. 7- As shown by the remapped cumula- ,S filter hL no DC component, Ae rSg^^ 

^£k?;^r^r^?- of histogram equal- 50 value is equally likely to ^poM^ve or^S^S 

^Si^J^ ^ ^^.^ positive results may be interpreted as pointf^ or near 

^ t£^^;,2tZjL*S ^nT^^'^^T °f ^^^^'^ "«8*t^« values repre- 

tSj^.^'Z^rSo'l"^^^. "°A^5ir:d^rs't''Sfo°^";hro b 

mg variation in the histograms ofFIGS./and7i,due 55 linS^^'^^ifod^Tn^el^'So^T^tateS"^ 

to sampling and quantization error in the picture data.) luncuon acsignaxea as. 

Basic Tonal Tranrfbnnation NctEffect(i,y)==NoaUiia«ity(Effec^^^ m 

A method of tonal correction of an image using cu- 
mulative histogram equalization of normalized filtered where NonUnearity may be any nonlinear function 
lummance values will now be described. This technique 60 which preserves the sign of Effect, and (x,y) are spatial 
mcorporates cumulative histogram remapping as previ- coordinates designating a pixel in the image. The cube 
ously descnbed; however, the histogram which is root fimction is advantageously used for the non- 
equalized IS not a standard histogram but rather is a linearity. The impact ofthe Nonlinearity fimction is to 
histogram whose values (counts) depend on the spatial even out the importance of large and medium sized 
frequency content of the picture as well as on its lumi- 65 gradients. 

'^f?'^^?!; . ^« NetEffect function has been computed. 

Stated briefly, filtered histogram equalization pro- three histograms arc made: a normal histogram of pixel 
ceeds as follows: counts, a histogram accumulating positive NetEffect 
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values (representing the bright side of edges), and a -continued 

histogram accumulating negative NetEfFect values LowEd e( = LowEdgefJO) (4) 

(representing the shadow side of edges). * ^ Norm(i)/NonLinearity(Nonn(£)) 

The normal histogram is computed in the standard 

way from L values of pixels from the image. For each 5 for each bin L in the histogram, where Norm(L) is the 

pixel of the image, a histogram bin corresponding to the normal histogram of pixel counts from the entire image, 

brighmess of that pixel is incremented. Once the normalization is complete, the resulting 

The computation of the edge histograms is similar but histograms are cumulated. A further adjustment is made 

uses NetEffect values. As with the normal histogram, ^ avoid excessive quantization which can happen if a 

for each pixel of the image, the histogram bin corre- very flat picture k processed. In order to avoid this 

spending to the brightness of that pixel is incremented. phenomenon, each cumulated lustogram ^ allowed to 

However, unlike the normal histogram, the size of the '^^'^ i^e than 4 parts m 256 from one histogram bm 

increment depends on the NetEffect function described ^^^^ t ^ ^ - ^ - ^ v 
t 1 ^»^«. ^ /• ^- . J The final tonal transfer is derived from a linear corn- 
above. If the NetEffect function is positive, the second , . " J""*" uouaici » uwivcu lium ^ 
, . , ir * 1 * 15 binaUon of the average of the high and low edge cumu- 
histogram is mcr«nented by an amount equal to the histograms. For each bri^ess B, the High and 
value of the NetEffect function at that pixel. If the t * uai^miw» 
^r^r^ wixiiicL,L xim^Liuii at XI LUC cumulative histogram counts are added and the 
NetEffect fiinction is negative, the tiurd hi^ogram is ^^^^ ^ ^ ^^^^ cmnulative histo- 
nicremented by an amount equal to the absolute value then hnearized, and the transformation which 
of the NetEffect function at that pixel. Thus, the two 20 linearizes the normalized histogram is applied to the 
"edge" histograms represent the total amount of me- image 
dium sized spatial detail which is present at each lumi- Colorfuhiess Transform 

nance level. Remapping brightness using the transformation de- 

After the three histograms are generated, the histo- scribed above can make objects in an image appear to 
grams may be "cleaned up" by removing spikes. This 25 become more or less colorful, because the colorfuhiess 
cleaning has two purposes. First, it cuts down on noise. of objects depends on their brightness. For many types 
Because the histograms nltimately will be combined in a of pictures, this side-effect is benefidal. For other pic- 
ratio, it is important that they be relatively free of noise, tures, including pictures which prominently feature 
The second purpose is to deal with situations where a people, this side effect is undesirable. Therefore, the 
disproportionately large number of points have occu- 30 system described here provides for a transformation to 
pied a single brightness bin. This situation is particularly be made to the chrominance values at each point which 
likely at the low and high ends of the brightness scal^ compensates for the change in colorfulness. Such a 
where the clustering is caused by out of range values in compensation transformation moves the chrominance 
the original photograph or scan. of a point closer to neutral if the point is being made^ 

The spike removal is done for each of the three histo- brighter, and moves the chrominance of a point farther^ 

grams by computing local thresholds based on average away from neutral if the point is being made darker, 

values around each histogram point. Each point of each Colorfulness is roughly proportional to the product 

histogram (i.e,, the pixel count or sum of NetEffect of brightness and saturation. One compensation trans 

values for a given bin) is clipped by this local threshold. formation moves the chrominance of each pixel 

The three histograms are thai smoothed. towards or away from neutral by an amount which is 

Stated another way, the smoothing process proceeds proportional to the change in brightness ^.e., to the 

as follows: ratio of the input and output brightnesses). However, 

Let: BrightnessHistogram=the histogram of L val- such a proportional transform tends to overcompensate 

ues. in situations where large tonal transforms are taking 

Let: SmoothFilter be a smoothing filter Smooth- Pl^e. Therefore, a more effective compensation takes 

Length points long. the average of the input chrominance and the result of 

For example, SmoothFUter may be given by Smooth- proportional method: 
Filter={l, 2, 3. 2. 1} (SmoothLength=5) 

Step 1 — Compute local thresholds; 50 • ' r/ "1^^ 

for each luminance bin i, look at the interval i- "X" *^ T f "^a^' + ("fa' - "wAft*') 

SmoothLength/2 toi+SmoothLength/2; -I 

compute the average of all points in the interval, 

excluding the biggest point in the interval; v ' - j^l-[f « j. JLfn , . * "I 

multiply the average by 2 to get a local threshold 55 - 2 + 2 l[J^^ + Lous " J 
value at i (this value 2.0 represents a tradeoff between 

letting noise in and keeping detail out). Because some pictures benefit from the distortion of 

Step 2~Clip the histogram usmg the local threshold. colorfulness, this variation is optional. 

Step 3 — Convolve the clipped histogram with the Gamut Compensation 

smoothing filter. 60 Though an unage corrected by the techniques de- 

After spike removal, the edge histograms (designated scribed above will have improved tonal characteristics 

"IfighEdge" for positive NetEffect values and "Lo- when represented in u'vX space, the tonality and color 

wEdge" for negative NetEffect values) are normalized reproduction may be distorted when output to a printer 

with an additional application of the non-linearity: (pt other device). Because printers cannot print all col- 

65 ors, all image processing systems which write to print- 

HighEdge(l.) <3) ©rs must apply an additional remapping which maps the 

HighEdge(L) = NonnCi^)/Noaliaearity(Noniud(L)) colors in an image into the printable gamut for that 

device. Therefore it is useful to adjust the tonal transfor- 
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ination in a way which allows for this gamut remap- luminance values, it can be implemented as a limit on 
P^^S- the basic tonal transform. That is to say, the gamut 
A typical gamut remapping needs to accomplish two compensation transform can be applied to modify the 
results. First, it needs to compress the tonal range of the operation of the basic tonal transform, rather than let- 
picture so that it fits within the printer's dynamic range. 5 ting the basic transform act first and then remapping the 
Second, it needs to modify colors which are too satu- pixels in a subsequent operation, 
rated to be printed. These results are typicaUy accom- The neutral compression compensation uniformly 
phsh^ by reducing the saturation of bright colors, by distributes the loss of contrast over different brightness 
modifymg the brightness Ouminance) of such colors, or levels of the picture. This can be visualized as a linear 
by some combmation of these two operations. lo input/output relationship shown as a straight line in 
Sainut compr^ion remappmgs can mt^ere piG. 10 where the slope of the line corresponds to the 
with the desired tonahty and color in two ways Firs^ ^etor A and the inter<!ept with the verticdscale is the 
the dynamic range compression will distort the desired offset B f ^ v« u«u *«uc i» mc 
tonal curve in a direct way. Second, in the case where Amoiepleasingresultseems to occur if sUghtly more 
tona^ remappmg causes a particular color to become 15 contrast Ic^ occiis in the darker areas of a p^etha^ 
^t^^'^l^T^^^ ^theli^terareas.T.e.for.animpro^^^ 
quence, an optimal tonal transform should not ^ 'tf''^ compression compensation formula uses an 
brighten satiLted colors too far beyond tSe^im ">P^t/output transfor^^^ 

wh«e they can be printed. ^"f ^^^y ^ l^ear transform 

It is desirable to construct a tonal transformation ^J^se slope at lar^brightnesses ^ 
which ultimately results in a printed image with desired ^ ^^^^ ^"^^^ ^ transform 
tonal qualities. An image processing apparatus which ^"^"^ {^L? ^"*P^ (Q^^ ColorScriptTO 100 
compensates for gamut transforms can be viewed as P™ter). The curve is a cubic splme which has a curva- 
performing a series of transformations on an image. slope at its top end. Insisting on unit 
First is a basic tonal move such as one using cumulative ^= ^ ^ over if the 
histogram equalization as described above. Then, a niimmum printable brightness is greater than 33 i; how- 
gamut-compensation transformation maps image realistic output systems do not come near this 
brightness values generated by the basic move into 

modified brightness values (as discussed in more detail 3^ Expressed another way: 

below). Finally, the apparatus maps the modified ^ . ^ 1 
brightness values into the printer gamut using a printing 
gamut transform. In practice, these transformations 

need not be serial, but can be combined into a sirigle ^^ere C is the curve shown in FIG. 10. 

comiK>sitetransfonnation.Howe^^^^^ 35 TT,e gamut compensation described above is applied 

standmz, gamut compensation transformations wiU be • 1 ji ^ ^ . -wyT** 40 ap^ucu 

discuss^ Ldividually b^w to aU pixels regardless of color. Further miprovements 

A first embodim Jt of a gamut compensation trans- ^ ^ made by separately tailoring the tonal compres- 

formation, referred to as neSSlcomprSncomS^^ A ^H^^^f . 

sation, will now be described As d^ussed abovVa ^ /^"""i embodment of a gamut compensation 

printing gamut transform maps colors from the u'v'L ^ tramformatioii referi^ to as chrominance dependant 

color space into a restricted Slor space which corre- will now be d^cussed. TTus transformation 

sponds to printable colors. Tlie inv«« of the Gamut ^l^"""^^ saturated colors from becommg so bright that 

transform (denoted as Gamut- 1) can be thought of as a unprmtable and . thus become desaturated by 

mapping of colors from the printable region back into ^1??*?^' 8^"* compression traiisform. 

the full space. The tonal compression compensation of method multiphes each pixel brightness value by 

this embodiment modifies luminance values produced * chrommanceKiependent attenuation fector. This at- 

by the basic tonal transform (denoted here as "Basic- tenuation factor depends on how bright a color with 

Move(L)") by a factor (denoted as "A*') to scale the chrominance can become without being strongly 

image luminance range. The range is then shifted by desaturated by the output gamut compression. Let 

adding an offset (denoted as "B'O so that the image Lmax,^, / be that brightest value, i.e., the value of L 

luminance values fall within the printable range. The gamut compression transform applied to 

scaled and shifted image luminance values are then the color (u', v', Lmaxu-y) produces a color change 

mapped back into full color space by applying the in- which is Itts than some predetermined limit. Methods 

verse Gamut transform. fcr determining Lmax«; f will be discussed in more 

The inverse Gamut transform operates on a point in detail below. The chrominance attenuation factor can 

the full color space, while BasicMove. (L) generates be set to Lmax«', //Lwhite. 

only a luminance value. When applying the inverse ^ exemplary apparatus applying such a chromi- 

Gamut transform, neutral chrominance values (denoted nance-dependant attenuation maintains a table of Lmax- 

^ ^N»yN) can be combined with the luminance value to (^v) values. After applying a basic tonal move, the 

form a point in color space. Stated another way, the ^ apparatus scales the luminance value (L) of each pixel 

gamut compression transform (denoted as GCorrec- according to the Lmax value for that pixel's chromi- 

tion(L)) can be expressed as; nance value (u,v). Denoted another way, the output 

lu minan ce (output L) resulting from the combined ap- 

GConf^3ca(L)^Gtmut-\uj^v^^ plication of the tonal transform (BasicMove) and a 

Move(L)+Jl) C7) 65 gamut compensation transform can be expressed as: 

While the gamut compensation transform has been output x.=Bask:Move<jC).Lmix,^,/Lwhite (g) 
described as if it were a sieparate transform which scales 
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for a plurality of tones to a transform relationship de- storing the pixel representation in memory of a pro- 
rived from the operator command. grammed computer; 

4. The method of claim 3 wherein the carve is a cubic modifying the pizel representation at least in part 
spline. according to a tonal transform producing colors 

5. A method for improving tone of an original image 5 outside the gamut; and 

comprising steps of: modifying the pixel representation at least in part 
generating a pixel representation of the origina] im- according to a compensation transform which 
age; modifies tone in relation to an amount of tone alter- 
fUtering the pixel representation with a filter having a ation imposed by the limited output gamut, thereby 
response peak near a spatial frequency at which the 10 generating an improved image, 
human is highly sensitive; 15. The method of claim 14 wherein each of the mod- 
computing an effect value for pixels based oh filtered ifying steps is performed sequentially. 

pixel values; 16. The method of daim 14 wherein two of the modi- 
generating a first histogram from pixels having effect fying steps are performed as a composite step using a 

values greater than a threshold; and 15 composite transform, 
generating a second histogram from pixels having 17. The method of claim 14 wherein the three modi- 
effect values less than the threshold; fying steps are performed as a composite step using a 
converting pixel tones in accordance with the first composite transform, 
and second histograms, thereby generating an im- 18. The method of claim 14 wherein the compensa- 
proved image. 20 tion transform comprises a linear moping of the tonal 

6. The method of claim 5 wherein a histogram gener- transform output in conjunction with an inverse map- 
ating step accumulates e£fect function values for pixels ping of the compression transform. 

in each of a series of brightness bins. 19. The method of claim 14 wherein the compensa- 

7. The method of claim 5 wherein a histogram gener- tion transform comprises a non-linear mapping of the 
ating step clips histogram values according to a local 25 tonal transform output in conjunction witfi an inverse 
threshold, thereby removing spikes. mapping of the compression transform, and the non-lin- 

8. The method of claim 5 wherein a histogram gener- ear mapping retains more contrast in brighter parts of 
ating step smooths histogram values. the image than in darker parts. 

9. The method of claim 5 further comprising a step of 20. The method of claim 14 wherein the compensa- 
generating a normal histogram of brightness of pixels in 30 tion transform modifies tone as a function of chromi- 
the pixel representation. nance. 

10. The method of claim 9 further comprising steps of 21. The method of claim 20 wherein the compensa- 
combining and cumulating the first, second and normal tion transform attenuates tone as a function of a maxi- 
histograms to generate a cumulated normalized histo- mum brightness below which the compression trans- 
gram. 35 form shifts color less than a significant amount. 

11. The method of claim 9 further comprising steps of 22. The method of claim 20 wherein the compensa- 
cumulating each of the first, second and normal histo- tion transform attenuates darker tones less than brighter 
grams and then combining the cimralated histograms tones. 

into a cumulated normalized histograna. 23. The method of claim 14 further comprising steps 

12. The method of claim 9 further comprising steps 40 of: 

of: displaying to an operator an image modified by the 

combining and cumulating the first, second and nor- basic and compensation transforms; 

mal histograms to generate a cumulated normal- receiving from an operator at least one command to 

ized histogram; and alter a tone selected from the displayed image; 

equalizing the cumulated and normalized histogram. 45 generating a modification transform which maps 

13. A method for improving tone of an original image image tones according to a transform relationship 
comprising the steps of: derived from the operator command; and 

generating a pixel representation of the original im- further modifying the pixel representation according 

age; to the modification transform, thereby generating a 

modi^ing pixel tones; and 50 further improved image. 

modifying pixel chrominance in accordance with the 24. The method of claim 23 wherein the transform 
amount of tone modification, there by generating a generating step fits a curve defining transform relation- 
tone-modified image with colorfulness compensa- ships for a plurality of tones to a transform relationship 
tion and wherein such tone modifying step in- derived from the operator command, 
eludes: 55 25. The method of claim 24 wherein the curve is a 

(i) fdtering the pixel representation with a filter cubic spline. 

having a frequency repsonse peak near a spatial 26. A method for improving tone of an original image 

frequency at which the human eye is highly and compensating for a lwni»f>H output gamut compris- 

sensitive; and ing steps of: 

(ii) converting tones of pixels to emphasize patterns 60 generating a pixel representation of the original im- 
in the original image having tone values which age; 

vary at a spatial frequency at which the human generating an initial tonal transform that emphasizes 

eye is highly sensitive. patterns in the original image having tone values 

14. A method for improving tone of an original image which vary at a spatial frequency at which the 
and compensating for tone alteration imposed by a lim- 65 human eye is highly sensitive, 

ited output gamut comprising steps of: displaying the pixd representation to an operator 

generating a pixel representation of the original im- receiving from the operator at least one command to 

age; alter a tone selected from the displayed image; 
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generating a manual transfonn which farther maps 
image tones according to the received command imtial and manual transforms, thereby generating 

while preserving the character of the initial tonal ^„ ; * 

transform; and ^ improved miage, 

modifying the pixel representation according to the 5 ♦ ♦ • ♦ » 
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Lmax can be determined directly from measm'ements put luminance to an input luminance. Different curves 

of a printer's characteristic. Alternately, Lmax can be are used for different Lmax values. That is to say, if 

determined from the printing gamut transform itself. Lmax for a particular chrommance (u,v) is 80% of full 

The gamut transform maps colors (u.v,L) which are not scale, the relationship of curve 1 is used. On the other 

printable into colors within the capabilities of a selected 5 hand, if Lmax is 60% of full scale, the relationship of 

prmter. The gamut transform maps chrominance and curve 2 is used. 

luminance values. The amount of chroma shift (the T-k * j -.t_ 1 ^* ^ • 

u^uuui * V «i wtuv/Luo ouui. yKu^ DcDotcd anothcr way, the relationship between 

difference between mput u,v and output u,v) can be * j *v j ^ j ^ • ^ u^^y^^^ 

thought of as a chroma error. The amount of error ^max^^ the modified chrommanccKlependant attenu- 

varies with luminance. That is, chroma error intro- m ation is a cubic spbnehavmg zero curvature at luminan- 

duced at low lummance will tend to be less than chroma ^ * ^^P^ maximum luminance which 

error introduced at higher luminance. The values for varies with Lmax: 
Lmax can be taken as a luminance at which the chroma 

error becomes large by some measure. Denoted another 2 (ip) 

way chroma shift introduced by the printer gamut x«,rO!:im«) = -r^^ . w f i.rfl--r^l H 

transform (denoted as ChromaError) can be expressed i^w/tae \^ unax j jj 
as: 

where: 

ChromaEtror=W.[(ii'-«'„^-(,/-v'«,^ (9) L is input luminance 

^^^^ 20 Lwhite is full luminance; and 

T 5 . 1 . 1 \_ ^ r .X. ^ the parameter t is a function of Lmax. 

L<wr IS the lummance value output from the gamut Ji **t-ij^t_ 

teansfoTm;aud The parameter t should be chosen so that the slope at 

(u'.uW and (v'-v'o«r) are the chrominance shifts "^^ff O-max^. Lwhite) is one and graduaUy becomes 

introduced by the gamut compression transform. ^^^^^ ^ ^' ^ *^ way the family of 

The value Lmax(u.v) can be taken to be a luminance ^ * regular way and each curve remains 

at which: a) the chroma error begins growing; b) the monotonic. A preferred in^Iementation uses the for- 
chroma error reaches a minimum value or 3) the 
chroma error exceeds a threshold, eg., 10-25% of the 

square of the saturation, uV (16% is preferred). 3 

The implementation of a chrominance-dependant j_ J_ Lmax 1 

attenuation has been described above as an apparatus ^ L J 

which maintains a look-up table of Lmax. It should be With this relationship, the slope at Lwhite. which gives 

understood that other implementations can be made for the maximum flattening of contrast is given by 
storing information about the printer characteristics. 

such as parametric descriptions, as long as some mea- slope (Lwhite)->(Linax/Lwiiite)4 
sure of printer limit is used. 

Chrominance-dependant attenuation, as discussed Such relationships can be implemented as an interpo- 

above. renormalizes the basic tonal remapping curve so lated table lookup or other method, 

that the top of the tonal curve goes to a brightness value Tlie gamut compensation transforms have been de- 

which is not too far above the upper limit for a printer. -«> scribed above for use with gamut compression trans- 

For a ^en chrominance, the tonal remapping is still forms which map unpintable colors into printable colors 

hnear TTiat is to say, ^e scale factor is the same in a controUed way. The gamut compensation trans- 

whether lummance is high or low. Whde this is an im- forms can also be used alone in systemswhich do not 

provement oyer chrommance-independent compensa- ^ ^ compression transforms. Without gamut 

v°l^'.r^'^^TT "".^"^^ com^tion Znsforms, output devices inhJently 
vantage of the fact that gamut compression tends to -n r -.1. • -iT^ * ,7 M*±M±^^uujf 
t i^^^r^ ^* ^i^i, I will chp or otherwise alter unpnntable colors into print- 
nave a greater impact at high luminance values (smce ^ui^ «J™ a ♦ 1 * c Ir- 
these are outside the printir^gamut). While it may be ^^^1 ^ ^^"^ .transform can anticipate such an 
appropriate to scale high luminance values by a factor "^^at^oUed distortion m the same manner described 
of Lmaxyl-white, this factor may unnecessarily darken 50 expressly designed gamut compensation 
colors with lower luminance values, such as saturated transforms, 
midtones. Operator Control 

A third embodiment of a gamut compensation trans- another embodiment of the current invention 

formation, referred to as modified chrominance depen- provides a mechanism for the operator to modify the 

dant attenuation, will now be discussed. Scaling is still a 35 transformations described above without changing their 

function of Lmax(u,v). However, the amount of scaling ^ocai character. The algorithms described above do not 

is not a linear factor. Instead, it is a function which recognize objects as such or incorporate any other type 

apportions more scaling to the higher luminance values of semantic analysis. Therefore, it is possible for them to 

and less scalmg to lower luminance values. Such a fimc- produce assignments of tonality which are not aestheti- 

tion continues to reach the same maximum attenuation 60 cally optimal. For this reasg i, an operator may desire to 

value (Lmax/Lwhite) when the pixel luminance is at adjust the output level fofOTFor mor'fe 6bj^)ETnnQtr " 

full scale. However, it provides a greater slope (less ^cture . However, even when this adjustment nccnn:^ it 

scalmg) in shadows and midtones. ifSSirable that the tendency of the transform to em- 

FIO. 11 is a graph depicting a family of attenuation phasize detail be retained. It is also desirable that the 
curves. Along the bottom of the graph arc L values (as 65 operator be able to modify the tonal curve without 
a percent of full scale) which denote pixel luminance losing the advantages of the gamut compeniition trans- 
resulting from a basic tonal move. Along the left verti- fonns. The method described here fulfills this goal and 
cal axis are output L values. Each curve relates an out- aUows the operator to control the shape of the tonal 
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curve without having to specify an excessive number of cation Ser. No. 07/983,154, entitled "Image Processing 

control points. Apparatus and Method with Intelligent Pixel Measure- 

The overall tone transformation can be expressed as: ment Cursor" (incorporated here by reference). The 

operator then enters a **to" value which is displayed on 

u,ut=Gcorr^,(Op*r.torCcrv.(B«icMov^))) (U) 5 the control panel. FIG. 12D reflects a value 46 ii^ut by 

^j^gj^. the operator. Operator input alternately may take the 

Lout is pixelluminance after a composite tonal move; ^l™ ^f. factional changes in tone (e.g *V or 

Lin is pixel luminance before a basic tonal move; ^ operator-s^phed mcrcment (e.g., plus 

BasicMovc is a tonal move; ^0% or plus 40 umts) or other command. 

Gcorr„,. is a gamut compensation transform; and ^ ^/^^"^ which result from the opera- 

OperatorCurvc is a discretionary transfonn mapping 5?^ requested m FIGS. 12A-D arc shown in FIG. 13. 

an input luminance to an output luminance. curves show the default and modified tonal curves 
Creation of an OperatorCurve transform has two ^® chrominance of the object which the operator 
parts. First, the operator inputs a scries of control ^ selected, qrhc object in this case was the arm of a 
points. Each control point specifies an input brightness toiuied Caucasian woman. The color was u'=0.252, 
level which the operator wishes to map to a specific v'=0.513, L=46). Because the example also mchides 
output brightness level. Second, the image processing gamut compensation transform, the TnaximuTn out- 
system fits a low order curve to the set of points (x/, y/) P^^' value is less than 100, The effect of the operator 
where ^ transform is to pull the fleshtones back up to their origi- 
nal brightness. Note, however, that over small bright- 
xf=Ba$icMove(Specificdl[iputd ness intervals, the local shape of the curve is not af- 

fected very much. 

y/=C3oiT- <SpedfiedOutPutd ^ ^^^^ understood that the appHcation of such 

where (Specifiedlnputr, SpecifiedOutput,) is the iU 25 °»^PP^f ^ ^ transformation of an original im- 

control W^hose^ by tlTSrator.V^^^ Sto m^alfe^eS^ 

embodiment this curve is a ci^ spline. Note that the P^^er to make the miage palaUble to the computer, the 

resulting curve is not a polynomial hi brightness, but is "^^^ *f expressed in a bmary equivalent form 

instead a polynomial in BasicMovc(brightaess). In this processing system. AppHcation of the 

way, the local curvature of the transformation remains 30 ^fo™ changes the amphtude of selected pixels and 

dominated by the BasicMovc. ^ ^ modified image still in binary form. The 

It is preferred that the system provide the operator "^^^ ^ printed, or transmitted dec- 

with a semi-automatic input process for inputting con- tromcally over a commumcation channel. Nevertheless, 

trol points. In this system, the operator places a mea- it is still an unagc. 

surement cufer'ove r a part oi a mctoct^^w^ appM rs"35 . Although a few preferred embodiments of the inven- 

to" be too ligh t or too jKl c i f e' operator thar^"^ shown and described, it will be appreci- 

opnon of specif ying a desired Value for th at point which skilled in the art that changes may be 

15 dlflSf gg wan themSsuSa valS TThe^so'lit^^ ^ ^^se embodiments without departing from the 

men iit^ubic^imethr o'ugli' tKe poin^ Wh)[<iM tt iFbt^cT ' principles and the spirit of the invention, the scope of 

ator Has defined. ' 40 which is defined in the appended claims. 

— '1 he system software displays control panels as shown What is claimed is: 

m FIGS. 12A-12D. FIG. 12A shows an initial control 1- A method for improving tone of an original image 

panel. By calling up the control panel, the image pro- comprising steps of: 

cessing apparatus automatically remaps the image with generating a pixel representation of the original im- 
a default tone transform, which may 1>e one or a combi- 45 

nation of tonal transforms described above. W hen the filtering the pixel representation with a filter having a 

control panel is activated^_Ae_apparatus_seleci,tiiLdaxl^^ response peak near a spatial frequency at which the 

SKpshadow"and~briglitest highlight from the image as"^ human eye is highly sensitive 

^default-controLpoints — ■ : — ^ converting pixel tones to emphasize patterns in the 

* — ^FIG.-12B-shows-a^control panel used to add a new 50 original image having tone values which vary at a 

point to the transform curve. When a control point is spatial frequency at which the human eye is highly 

initially added, its de&ult **From" value is midway sensitive, diereby generating an improved image, 

between the "From" values of adjacent control points; 2. The method of clahn 1 wherein the converting step 

thus, for the first point added, the default '"From" value comprises steps of: 

will be midway between the shadow and highlight val- 55 modifying the pixel representation according to a 

ues. The *To" value of the new point will default to the tonal transform; 

existmg transform applied to the **From** value. Thus, displaying to an operator the modified pixel represen- 

simply adding a control point without changing the tation; 

"From" or "To" value does not by itself change the receiving from an operator at least one command to 

transform. 60 alter a tone selected from the displayed representa- 

FIG. 12c shows the use of a "Measure** function tion; 

selectable from the control panel. The operator may use generating a modification transform which maps 

a mouse or other input device to position a measure- image tones according to the operator command; 

ment cursor on a portion of the image and then activate and 

the Measure function to update the "From" value of the 65 modifying the image according to the modification 
point to the luminance value of the portion of the image transform, thereby generating an improved image, 
under the measurement cursor. Such a capability is 3. The method of claim 2 wherein the transform gen- 
described in more detail in copending U.S. patent appli- crating step fits a curve defining transfonn relationships 
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